Solid-state image pickup device and image pickup method, and electronic apparatus

ABSTRACT

The present disclosure relates to a solid-state image pickup device that is adapted to enable the light exposure time to be shortened, and an image pickup method, and an electronic apparatus. 
     One pixel and the other pixel differ in the timing in which light exposure is started and in the timing in which the light exposure is ended. During the light exposure time of the one pixel, active light starts light emission, and completes the light emission. For example, before or after the light emission of the active light, the one pixel starts light exposure, and ends the light exposure. The other pixel starts light exposure in the timing in which the time Ta 1  has passed after the active light starts light emission (the time Ta 2  until the end of irradiation remains), and ends the light exposure after the light exposure of the one pixel ends. The present disclosure can be applied to, for example, an image pickup device that performs image pickup by using active light.

TECHNICAL FIELD

The present disclosure relates to a solid-state image pickup device andan image pickup method, and an electronic apparatus, and in particular,to a solid-state image pickup device that is adapted to enable the lightexposure time to be shortened, and an image pickup method, and anelectronic apparatus.

BACKGROUND ART

Patent document 1 proposes a technology in which even in a case where animage is taken while being irradiated with infrared light, visible lightand infrared light can be properly separated from an image signal thatincludes the visible light and the infrared light.

In this proposal, by irradiating with infrared light over two frames, adifference in light exposure time is used to determine the visible lightand the infrared light by computation processing.

CITATION LIST Patent Document

Patent Document 1: Japanese Patent Application Laid-Open No. 2016-76807

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, in the case of the proposal of the patent document 1, theirradiation is performed over two frames, and therefore long LED lightemission time is required.

The present disclosure has been devised in light of such circumstances,and is capable of shortening the light exposure time.

Solutions to Problems

A solid-state image pickup device according to one aspect of the presenttechnology includes: a pixel array unit on which pixels aretwo-dimensionally arranged; and a light exposure control unit thatcontrols light exposure time of a first pixel, and light exposure timeof a second pixel that differs in light exposure time from the firstpixel in such a manner that at least one of light exposure start time orlight exposure end time differs according to light emission time ofpredetermined light in the pixel array unit.

The light exposure control unit is capable of controlling the lightexposure time of the first pixel and the light exposure time of thesecond pixel in such a manner that the light exposure start time differsaccording to the light emission time of the predetermined light.

The light exposure control unit is capable of controlling the lightexposure time of the first pixel and the light exposure time of thesecond pixel in such a manner that the light exposure start time differsby changing an inclusion ratio of the light emission time of thepredetermined light.

The pixel is capable of including a PD.

The light exposure control unit is capable of controlling the lightexposure start time by reset operation, and is capable of controllingthe light exposure end time by charge transfer.

The pixel is capable of including an organic or inorganic organicphotoelectric conversion film.

The light exposure control unit is capable of controlling the lightexposure start time by reset operation, and is capable of controllingthe light exposure end time by an upper electrode of the photoelectricconversion film.

The light exposure control unit is capable of controlling the lightexposure start time of at least one of the first pixel or the secondpixel by an upper electrode of the photoelectric conversion film, and iscapable of controlling the light exposure end time by the upperelectrode of the photoelectric conversion film.

The light exposure control unit is capable of controlling the lightexposure time of the first pixel and the light exposure time of thesecond pixel in such a manner that at least one of the light exposurestart time or the light exposure end time differs according to lightemission times of a plurality of the predetermined lights.

The light exposure control unit is capable of controlling the lightexposure time of the first pixel and the light exposure time of thesecond pixel in such a manner that at least one of the light exposurestart time or the light exposure end time differs by changing inclusionratios of the light emission times of a plurality of the predeterminedlights respectively.

The solid-state image pickup device is capable of further including acomputation unit that subjects images from the first pixel and thesecond pixel to mosaic processing, and performs computation processingon a pixel basis.

The light exposure control unit is capable of controlling the lightexposure time of the first pixel, the light exposure time of the secondpixel, and the light exposure time of a third pixel that differs inlight exposure time from the first pixel and the second pixel in such amanner that at least one of the light exposure start time or the lightexposure end time differs according to light emission times of aplurality of the predetermined lights.

The light exposure control unit is capable of controlling the lightexposure time of the first pixel, the light exposure time of the secondpixel, and the light exposure time of the third pixel in such a mannerthat at least one of the light exposure start time or the light exposureend time differs by changing inclusion ratios of the light emissiontimes of a plurality of the predetermined lights respectively.

The solid-state image pickup device is capable of further including acomputation unit that subjects images from the first pixel, the secondpixel, and the third pixel to mosaic processing, and performscomputation processing on a pixel basis.

The pixel array unit is capable of including a pixel having a conversionefficiency adjustable function.

An image pickup method according to one aspect of the present technologyincludes the step of controlling, by a solid-state image pickup device,light exposure time of a first pixel, and light exposure time of asecond pixel that differs in light exposure time from the first pixel insuch a manner that at least one of light exposure start time or lightexposure end time differs according to light emission time ofpredetermined light in a pixel array unit on which pixels aretwo-dimensionally arranged.

An electronic apparatus according to one aspect of the presenttechnology includes: a light-emitting unit that emits light; and

a solid-state image pickup element, the solid-state image pickup elementincluding: a pixel array unit on which pixels are two-dimensionallyarranged; and a light exposure control unit that controls light exposuretime of a first pixel, and light exposure time of a second pixel thatdiffers in light exposure time from the first pixel in such a mannerthat at least one of light exposure start time or light exposure endtime differs according to light emission time of light emitted by thelight-emitting unit in the pixel array unit.

According to one aspect of the present technology, in the pixel arrayunit on which pixels are two-dimensionally arranged, the light exposuretime of the first pixel, and the light exposure time of the second pixelthat differs in light exposure time from the first pixel are controlledin such a manner that at least one of the light exposure start time orthe light exposure end time differs according to the light emission timeof predetermined light.

Effects of the Invention

According to the present technology, the light exposure time can becontrolled. In particular, according to the present technology, thelight exposure time can be shortened.

It should be noted that the effects described in the present descriptionare to be construed as merely illustrative, and that effects of thepresent technology are not limited to those described in the presentdescription, and thus an additional effect may be produced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a schematic configuration exampleof a solid-state image pickup device to which the present technology isapplied.

FIG. 2 is a diagram illustrating two kinds of light exposure controlrelated to active light and pixels.

FIG. 3 is a diagram illustrating computation processing after output.

FIG. 4 is a diagram illustrating an example of light exposure operationof pixels in the case of a global shutter.

FIG. 5 is a diagram illustrating an example of light exposure operationof pixels in the case of a rolling shutter.

FIG. 6 is a diagram illustrating, as an example, a circuit configurationand the light exposure control timing in the case of a global shutter.

FIG. 7 is a diagram illustrating, as an example, another circuitconfiguration and the light exposure control timing in the case of aglobal shutter.

FIG. 8 is a diagram illustrating, as an example, still another circuitconfiguration and the light exposure control timing in the case of aglobal shutter.

FIG. 9 is a diagram illustrating, as an example, a further circuitconfiguration and the light exposure control timing in the case of aglobal shutter.

FIG. 10 is a diagram illustrating an example of a circuit configurationin the case of a global shutter that uses a photoelectric conversionfilm.

FIG. 11 is a diagram illustrating an example of the light exposurecontrol timing in the case of FIG. 10.

FIG. 12 is a diagram illustrating an example of another circuitconfiguration in the case of a global shutter that uses a photoelectricconversion film.

FIG. 13 is a diagram illustrating an example of the light exposurecontrol timing in the case of FIG. 12.

FIG. 14 is a diagram illustrating two kinds of light exposure controlrelated to two active light beams and pixels.

FIG. 15 is a diagram illustrating three kinds of light exposure controlrelated to two active light beams and pixels.

FIG. 16 is a diagram illustrating computation processing after output.

FIG. 17 is a diagram illustrating, as an example, a circuitconfiguration and the light exposure control timing in the case of aglobal shutter.

FIG. 18 is a diagram illustrating an example of a pixel pattern.

FIG. 19 is a diagram illustrating another example of a pixel pattern.

FIG. 20 is a diagram illustrating an example of another pixel pattern.

FIG. 21 is a diagram illustrating an example of a physical image ofcontrol lines.

FIG. 22 is a diagram illustrating another example of a physical image ofcontrol lines.

FIG. 23 is a diagram illustrating still another example of a physicalimage of control lines.

FIG. 24 is a diagram illustrating an example of another pixel area.

FIG. 25 is a diagram illustrating an example that includes a conversionefficiency adjustable function.

FIG. 26 is a diagram illustrating an example of using an image sensor towhich the present technology is applied.

FIG. 27 is a block diagram illustrating a configuration example of anelectronic apparatus to which the present technology is applied.

FIG. 28 is a diagram illustrating an example of a schematicconfiguration of an endoscopic operation system.

FIG. 29 is a block diagram illustrating an example of a functionalconfiguration including a camera head and a CCU.

FIG. 30 is a block diagram illustrating an example of a schematicconfiguration of a vehicle control system.

FIG. 31 is an explanatory diagram illustrating an example of positionsat which a vehicle outside information detection unit and an imagepickup unit are provided.

MODE FOR CARRYING OUT THE INVENTION

Modes for carrying out the present disclosure (hereinafter referred toas embodiments) will be described below. It should be noted thatexplanations are made in the following order.

0. Explanation of device

1. First embodiment

2. Second embodiment

3. Third embodiment

4. Example of using image sensor

5. Example of electronic apparatus

6. Example of application to endoscopic operation system

7. Example of application to moving object

0. Explanation of device Schematic Configuration Example of Solid-StateImage Pickup Device

FIG. 1 illustrates, as an example, a schematic configuration example ofa Complementary Metal Oxide Semiconductor (CMOS) solid-state imagepickup device that is applied to each embodiment of the presenttechnology.

As shown in FIG. 1, a solid-state image pickup device (element chip) 1includes a semiconductor substrate 11 (for example, a silicon substrate)having a pixel region (so-called an image-pickup region) 3 in whichpixels 2 each including a plurality of photoelectric conversion elementsare regularly and two-dimensionally arranged, and a peripheral circuitregion.

The pixel 2 includes a photoelectric conversion element (for example,Photo Diode (PD)), and a plurality of pixel transistors (so-called MOStransistors). Three transistors, which are, for example, a transfertransistor, a reset transistor, and an amplification transistor,constitute the plurality of pixel transistors. The plurality of pixeltransistors can also be constituted of four transistors by furtheradding a selection transistor.

In addition, the pixel 2 can also have a pixel sharing structure. Thepixel sharing structure includes a plurality of photodiodes, a pluralityof transfer transistors, one shared floating diffusion, and other pixeltransistors shared one by one. The photodiodes are photoelectricconversion elements.

The peripheral circuit region includes a vertical driving circuit 4,column signal processing circuits 5, a horizontal driving circuit 6, anoutput circuit 7, and a control circuit 8.

The control circuit 8 receives an input clock, and data that gives aninstruction on an operation mode or the like, and outputs data such asinternal information of the solid-state image pickup device 1. Morespecifically, on the basis of a vertical synchronizing signal, ahorizontal synchronizing signal, and a master clock, the control circuit8 generates a clock signal that functions as a reference of theoperations of the vertical driving circuit 4, the column signalprocessing circuits 5, and the horizontal driving circuit 6, andgenerates a control signal. Subsequently, the control circuit 8 inputsthese signals into the vertical driving circuit 4, the column signalprocessing circuits 5, and the horizontal driving circuit 6.

The vertical driving circuit 4 includes, for example, a shift register,selects a pixel driving wiring line, supplies the selected pixel drivingwiring line with a pulse for driving the pixel 2, and drives the pixel 2on a line basis. More specifically, the vertical driving circuit 4sequentially selects and scans each of the pixels 2 in the pixel region3 on a line basis in the vertical direction, and supplies, through avertical signal line 9, each of the column signal processing circuits 5with a pixel signal based on a signal charge generated according to theamount of received light in the photoelectric conversion element of eachof the pixels 2.

The column signal processing circuits 5 are arranged corresponding to,for example, respective columns of the pixels 2, and each subjectsignals output from the pixels 2 in one column to signal processing suchas noise removal on a pixel column basis. More specifically, the columnsignal processing circuits 5 each perform signal processing such asCorrelated Double Sampling (CDS) for removing fixed pattern noisesspecific to the pixels 2, signal amplification, and Analog/Digital (A/D)conversion. A horizontal selection switch (not illustrated) is connectedand provided between an output stage of each of the column signalprocessing circuits 5 and a horizontal signal line 10.

The horizontal driving circuit 6 includes, for example, a shiftregister, and sequentially outputs a horizontal scanning pulse to selecteach of the column signal processing circuits 5 in order, which causesthe each of the column signal processing circuits 5 to output a pixelsignal to the horizontal signal line 10.

The output circuit 7 subjects a signal sequentially supplied from eachof the column signal processing circuits 5 through the horizontal signalline 10 to signal processing, and then outputs the signal. There is alsoa case where the output circuit 7 performs, for example, buffering only;and there is also a case where the output circuit 7 performs black leveladjustment, column dispersion correction, various digital signalprocessing, or the like.

The input/output terminal 12 is provided so as to exchange signals withthe outside.

1. First Embodiment Light Exposure Control of Pixels

FIG. 2 is a diagram illustrating the light emission timing (the lightemission time) of active light and the light exposure timing (the lightexposure time) of two kinds of pixels. The active light is, for example,IR light, monochromatic light, white light, or the like, but is notlimited thereto.

A pixel 2-1 and a pixel 2-2 are pixels that differ in the light exposurestart timing (the light exposure start time) and the light exposure endtiming (the light exposure end time) from each other. The pixel 2-1performs long exposure operation, the light exposure time T1 of which islong. During the light exposure time of the pixel 2-1, the active lightstarts light emission, and completes the light emission. For example, inthe example of FIG. 2, before or after the light emission of the activelight, the pixel 2-1 starts light exposure, and ends the light exposure.The pixel 2-2 performs short exposure operation, the light exposure timeT2 of which is shorter than the light exposure time T1 of the pixel 2-1,in the timing in which the time Ta1 has passed after the active lightstarts light emission (the time Ta2 until the end of irradiationremains). It should be noted that if the light exposure start timing ofthe pixel 2-2 is the timing in which the time Ta1 has passed after theactive light starts light emission, the light exposure time may be thesame as that of the pixel 2-1.

On the assumption that a signal of the pixel 2-1 is out1, a signal ofthe pixel 2-2 is out2, intensity of reflected light by active light isal, intensity of background light is bg, as shown in FIG. 2, the lightexposure time of the pixel 2-1 is T1, the light exposure time of thepixel 2-2 is T2, and during the light emission time of active light, thetime until light exposure of the pixel 2-2 starts is Ta1, equation (1)is obtained.

[Equation 1]

out1=bg×T1+a1×(Ta1+Ta2) out2=bg×T2+a1×Ta2  (1)

The reflected light intensity a1 by active light, and the backgroundlight intensity bg, can be determined from the above-described equation(1), and therefore it is possible to reduce the influence of reflectedlight.

In particular, in a case where Ta2 is 0, in other words, when the activelight does not overlap the light exposure time of the pixel 2, theequation (1) is remarkably simplified. It should be noted that in a casewhere a ratio between the light exposure time T1 and the time Ta1+thetime Ta2 is equivalent to a ratio between the light exposure time T2 andthe time Ta2, calculation is difficult. Therefore, it is necessary touse values that are not equivalent to each other. In other words, thetime Ta1 may be 0 (=0), or the time Ta2 may be 0 (=0). This is becausein a case where the time Ta1=0, and a value of the light exposure timeT1 is equivalent to that of the light exposure time T2, the ratios areequivalent to each other, which makes the calculation difficult.

It should be noted that although the pixels that differ in the lightexposure time may each have the same structure, the conversionefficiency and size thereof can be changed in light of the saturatedamount.

FIG. 3 is a diagram illustrating computation processing after output.

An example of FIG. 3 shows an output image 21 obtained in a case wherethe pixel 2-1 and the pixel 2-2 that (are controlled so as to) differ inthe light exposure timing, in other words, the pixel 2-1 and the pixel2-2 that (are controlled so as to) differ in ratio of included (input)active light, are arranged, for example, in a checkered pattern in thepixel area 3. As shown in an image 22-1 and an image 22-2, a computationunit 23 subjects this output image 21 to de-mosaic processing, andperforms computation processing on a pixel basis, thereby enabling alight exposure image 24 that is exposed to active light, and from whichthe influence of background light is eliminated to be obtained.

As described above, computation processing can be carried out by usingthe feature of differing in ratio between background light and activelight included in two (or a plurality of) signals. This enablesone-frame complete operation, and eliminates the need for a framememory. Further, the electric power is also low in comparison withtwo-frame operation.

Incidentally, with respect to a solid state image pickup element that iscapable of global shuttering, all pixels share the same computingequation, and thus the computing equation is simplified. Therefore, veryeffective operation can be achieved as described next.

Light Exposure Operation of Pixels in the Case Of Global Shutter

FIG. 4 is a diagram illustrating light exposure operation of pixels inthe case of a global shutter. As with the example of FIG. 2 describedabove, A of FIG. 4 shows an example of operation obtained in a casewhere the light exposure start timing and the light exposure end timingare shifted.

With respect to the pixel 2-1, light exposure is started before lightemission of active light is started, and the light exposure is endedafter the light emission of the active light ends. Meanwhile, withrespect to the pixel 2-2, light exposure is started during lightemission of active light, and the light exposure is ended after thelight exposure of the pixel 2-1 ends. Subsequently, reading from thepixel is performed.

Incidentally, the light exposure time is a parameter that is determinedby, for example, the synchronization timing of the active light, or thelight emission time of the active light capable of taking SN of thesignal. Therefore, by, for example, the control circuit 8 or the like ofFIG. 1, the timing can be controlled in accordance with an applicationto which the present technology is applied.

B of FIG. 4 shows an example of operation obtained in a case where thelight exposure end timings are kept aligned with each other.

In the case of the example of B of FIG. 4, the light exposure starttimings of the pixel 2-1 and the pixel 2-2 are similar to those of A ofFIG. 4 respectively. However, the light exposure of the pixel 2-1 andthat of the pixel 2-2 are ended in the same timing after the lightemission of the active light ends.

In other words, since the light exposure time of the pixel 2-2 overlapsthe light exposure time of the pixel 2-1, the total light exposure timeis shortened. Therefore, the light emission time can be shortened. Inaddition, high adaptability to a moving object is achieved. Moreover, aframe rate improving effect produced by a decrease in useless lightexposure time is expected. However, since all pixels are concurrentlydriven, operation that changes ending of light exposure in some casescan also be supported as appropriate.

Next, rolling shutter operation will be described. In the case of arolling shutter, light emission of active light may overlap the pixel2-1 on the pixel 2-2 side, or may not overlap the pixel 2-1. Inaddition, in the case of the rolling shutter, a coefficient forcomputation differs on a row basis, and therefore it is disadvantageousin that post-processing becomes complicated. However, since the pixelsdo not require a global shutter function, a configuration is simplified.

Light Exposure Operation of Pixels in the Case of Rolling Shutter

FIG. 5 is a diagram illustrating light exposure operation of pixels inthe case of a rolling shutter. A of FIG. 5 shows an example of operationperformed in a case where active light enters only the pixel 2-1.

In other words, with respect to the pixel 2-1, light exposure is startedbefore light emission of active light is started, and the light exposureis ended after the light emission of the active light ends. Meanwhile,with respect to the pixel 2-2, light exposure is started after lightemission of active light, and the light exposure is ended after thelight exposure of the pixel 2-1 ends. Subsequently, reading from thepixel is performed.

In the case of the example of A of FIG. 5, even if the light exposuretimes of active light are not kept aligned in all areas, calculation canbe performed. In addition, the light exposure time of active lightdiffers in the pixel 2-1, and therefore computation differs on a Rowbasis.

B of FIG. 5 shows an example of operation performed in a case whereactive light enters both the pixel 2-1 and the pixel 2-2.

As with A of FIG. 5, with respect to the pixel 2-1, light exposure isstarted before light emission of active light is started, and the lightexposure is ended after the light emission of the active light ends.Meanwhile, differently from the case of A of FIG. 5, with respect to thepixel 2-2, light exposure is started during light emission of activelight, and the light exposure is ended after the light exposure of thepixel 2-1 ends. Subsequently, reading from the pixel is performed.

In this example, in a case where a part is adding operation, asdescribed above, it is necessary to set the pixel 2-1 and the pixel 2-2so as to differ in ratio between the active light and the total lightexposure time. In a case where light enters the pixel 2-1 and the pixel2-2, a computing equation differs on a Row basis.

Circuit Configuration and the Light Exposure Control Timing in the Caseof Global Shutter

FIG. 6 shows, as an example, a circuit configuration and the lightexposure control timing in the case of a global shutter. A of FIG. 6shows, as an example, a two-pixel sharing circuit configuration in whicha general-purpose embedded PD method is used, the light exposure starttime in real operation is reset operation, and ending of light exposureis performed by charge transfer.

In other words, the pixel 2-1 and the pixel 2-2 each include an OFGtransistor 31, a TRX transistor 32, a TRG transistor 33, a photodiode(PD) 34, and a capacitor 35. However, a RST transistor 36, an AMPtransistor 37, an SEL transistor 38, and a floating diffusion (FD) 39are shared by the pixel 2-1 and the pixel 2-2.

In the pixel 2-1, as shown in B of FIG. 6, a TRX1 signal is input intothe TRX transistor 32 to perform reset operation, which causes lightexposure to start. The light exposure is ended by charge transfer.Meanwhile, in the pixel 2-2, a TRX2 signal is input into the TRXtransistor 32 to perform reset operation, which causes light exposure tostart. The light exposure is ended by charge transfer.

Incidentally, a pulse (control signal) is shown as an example.Therefore, an example in which the timing of light exposure control iscontrolled by an OFG signal as described below with reference to FIG. 7is also considered.

The circuit configuration shown in A of FIG. 7 is basically similar tothe circuit configuration shown in A of FIG. 6, and therefore thedescription thereof will be omitted.

In the pixel 2-1, as shown in B of FIG. 7, an OFG1 signal is input intothe OFG transistor 31 to perform reset operation, which causes lightexposure to start. The light exposure is ended by charge transfer.Meanwhile, in the pixel 2-2, an OFG2 signal is input into the OFGtransistor 3 to perform reset operation, which causes light exposure tostart. The light exposure is ended by charge transfer.

It should be noted that although the two-pixel sharing circuitconfiguration is shown in the examples of FIGS. 6 and 7, no circuitsharing or four-pixel sharing or the like can also be supported. Inaddition, although one TRX configuration is shown in the example of FIG.6, there can also be considered a method in which a plurality of gates(TRX1-1, TRF1-2, etc.) are arranged.

FIG. 8 shows, as an example, another circuit configuration and the lightexposure control timing in the case of a global shutter.

The circuit configuration shown in A of FIG. 8 differs from the circuitconfiguration shown in A of FIG. 6 in that the OFG transistor 31 isexcluded from the pixel 2-1 and the pixel 2-2.

In other words, the pixel 2-1 and the pixel 2-2 each include the TRXtransistor 32, the TRG transistor 33, the PD 34, and the capacitor 35.However, the RST transistor 36, the AMP transistor 37, the SELtransistor 38, and the FD 39 are shared by the pixel 2-1 and the pixel2-2.

In the pixel 2-1, as shown in B of FIG. 8, the TRX1 signal is input intothe TRX transistor 32 to perform reset operation, which causes lightexposure to start. The light exposure is ended by charge transfer.Meanwhile, in the pixel 2-2, a TRX2 signal is input into the TRXtransistor 32 to perform reset operation, which causes light exposure tostart. The light exposure is ended by charge transfer.

In this case, since there is no OFG transistor, the flexibility of thelight exposure control of the PD 34 decreases. However, the number ofthe transistors can be reduced.

It should be noted that although the two-pixel sharing circuitconfiguration is shown in the example of FIG. 8, no circuit sharing orfour-pixel sharing or the like can also be supported.

FIG. 9 shows, as an example, still another circuit configuration and thelight exposure control timing in the case of a global shutter.

As shown in A of FIG. 9, the pixel 2-1 and the pixel 2-2 each includethe TRG transistor 33, the PD 34, the RST transistor 36, the AMPtransistor 37, the SEL transistor 38, and the FD 39. In other words, thecircuit configuration shown in A of FIG. 9 is an example of a globalshutter method in which an electrical charge from the PD 34 istransferred to the FD 39. Therefore, the circuit configuration shown inA of FIG. 9 is not capable of pixel sharing.

In this case as well, in the pixel 2-1, as shown in B of FIG. 9, a TRG1signal is input into the TRG transistor 32 to perform reset operation,which causes light exposure to start. The light exposure is ended bycharge transfer. Meanwhile, in the pixel 2-2, a TRX2 signal is inputinto the TRX transistor 32 to perform reset operation, which causeslight exposure to start. The light exposure is ended by charge transfer.

FIG. 10 shows an example of a circuit configuration in the case of aglobal shutter that uses an organic photoelectric conversion film or aninorganic photoelectric conversion film (hereinafter merely referred toas photoelectric conversion film). It should be noted that in thecircuit configuration of FIG. 10, VC is a power supply voltage that isconnected to a transparent electrode 42, and VR is a power supplyvoltage that is connected to the RST transistor 36.

The circuit configuration of FIG. 10 differs from the circuitconfiguration of FIG. 9 in that in the pixel 2-1 and the pixel 2-2, thePD 34 is replaced with a photoelectric conversion film 41, thetransparent electrode 42, and a lower electrode 43, and the TRGtransistor 33 is removed.

In other words, as shown in FIG. 10, the pixel 2-1 and the pixel 2-2each include the photoelectric conversion film 41, the transparentelectrode 42, the lower electrode 43, the RST transistor 36, the AMPtransistor 37, the SEL transistor 38, and the FD 39. It should be notedthat in the pixel 2-1 and the pixel 2-2, each of which has thephotoelectric conversion film 41, for example, the photoelectricconversion film 41 controls a voltage of the transparent electrode 42,thereby realizing a global shutter.

In the case of the circuit configuration of FIG. 10, the following threelight exposure start controls can be performed. Incidentally, startingand ending of VC input into the transparent electrode 42 is controlledby controlling a voltage of the transparent electrode 42.

In the case of A of FIG. 11, an example in which the light exposurestart control is performed by the transparent electrode 42 is shown. Inthe pixel 2-1, light exposure is started from the timing in which theVC1 is input into the transparent electrode 42, and the light exposureis ended in the timing in which the inputting is ended. In the pixel2-2, light exposure is started from the timing in which the VC2 is inputinto the transparent electrode 42, and the light exposure is ended inthe timing in which the inputting is ended.

In the case of B of FIG. 11, an example in which the light exposurestart control is performed by a RST signal is shown. In other words, inthe pixel 2-1, light exposure is started in the timing in which the RSTsignal is input into the RST transistor 36, and the light exposure isended in the timing in which inputting of the VC1 is ended. In the pixel2-2, light exposure is started in the timing in which the RST signal isinput into the RST transistor 36, and the light exposure is ended in thetiming in which inputting of the VC2 is ended.

In the case of C of FIG. 11, an example in which electrodes are the samewith the VC1 and the VC2 simultaneously controlled or used as VC isshown. In other words, in the pixel 2-1, light exposure is started inthe timing in which the RST signal is input into the RST transistor 36,and the light exposure is ended in the timing in which inputting of theVC is ended. In the pixel 2-2, light exposure is started in the timingin which the RST signal is input into the RST transistor 36, and thelight exposure is ended in the timing in which inputting of the VC isended.

FIG. 12 shows an example of another circuit configuration in the case ofa global shutter that uses a photoelectric conversion film. In otherwords, FIG. 12 shows an example of a circuit configuration used in acase where a circuit is shared.

In other words, the pixel 2-1 and the pixel 2-2 each include thephotoelectric conversion film 41, the transparent electrode 42, thelower electrode 43, and the TRG transistor 33. However, the RSTtransistor 36, the AMP transistor 37, the SEL transistor 38, and the FD39 are shared by the pixel 2-1 and the pixel 2-2. It should be notedthat in the case of FIG. 12, although the transparent electrode 42 isillustrated in the same terminal as the VC, the transparent electrode 42may be physically separated.

In the case of A of FIG. 13, in the pixel 2-1, light exposure is startedin the timing in which a TRG signal is input into the TRG transistor 33,and the light exposure is ended in the timing in which inputting of theVC is ended. In the pixel 2-2, light exposure is started in the timingin which the TRG signal is input into the TRG transistor 33, and thelight exposure is ended in the timing in which inputting of the VC isended.

It should be noted that as with the example shown in A of FIG. 13, theRST transistor 36 into which the RST signal is input may be always ONduring a time period other than reading, or may be driven in a pulsedmanner so as to cover an ON time period during which the TRG1 signal andthe TRG2 signal are input.

In the case of B of FIG. 13, in the pixel 2-1, light exposure is startedfrom the timing in which the VC1 is input into the transparent electrode42, and the light exposure is ended in the timing in which the inputtingis ended. In the pixel 2-2, light exposure is started in the timing inwhich the TRG signal is input into the TRG transistor 33, and the lightexposure is ended in the timing in which inputting of the VC is ended.

In other words, B of FIG. 13 shows an example in which the lightexposure start timing of one pixel is realized by a transparentelectrode.

Incidentally, in the above description, the example in the case of theglobal shutter has been described. However, the rolling shutter can besupported by ordinary pixels, and performs general pulse operation, andtherefore the description thereof will be omitted. A difference betweenthe rolling shutter and the global shutter depends on whether or not allpixels are simultaneously exposed to light. In the case of the rollingshutter as well, the same waveform as that of the global shutter isobtained by operation of successively performing light exposure on a rowbasis.

2. Second Embodiment Light Exposure Control of Pixels

FIG. 14 is a diagram illustrating the light emission timing of two kindsof active light and the light exposure timing of two kinds of pixels.

Other than the light exposure control of pixels described with referenceto FIG. 2, there can also be considered a method in which what isdesired to be detected is detected from a difference in absorptioncoefficient by using light having two wavelengths. A change inconcentration of, for example, oxyhemoglobin, deoxyhemoglobin, or thelike can also be calculated.

In the method that uses two wavelengths, a value cannot be determinedonly by two kinds of light exposure. Active light 61 and active light 62are controlled in such a manner that only either of the active light 61and the active light 62 enters the pixel 2-1 or the pixel 2-2. Forexample, in the example of FIG. 14, before or after the light emissionof the active light 61, the pixel 2-1 starts light exposure, and endsthe light exposure. After the light emission of the active light 61ends, and before or after light emission of the active light 62, thepixel 2-2 starts light exposure, and ends the light exposure. It shouldbe noted that in the case of this FIG. 14 as well, the light exposureend timings in the respective pixels 2-1 and 2-2 may be the same as eachother.

As the result, a background can be subtracted. In this case, the outputof the active light 61 and the output of the active light 62 cannot beseparated from each other. However, a difference therebetween can becalculated.

On the assumption that a signal of the pixel 2-1 is out1, a signal ofthe pixel 2-2 is out2, intensity of reflected light by the active light61 is a161, intensity of reflected light by the active light 62 is a162,intensity of background light is bg, as shown in FIG. 14, the lightexposure time of the pixel 2-1 is T1, the light exposure time of thepixel 2-2 is T2, the light emission time of the active light 61 is Ta61,and the light emission time of the active light 62 is Ta62, thefollowing equation (2) is obtained.

[Equation 2]

out1=bg×T1+a11×Ta61 out2=bg×T2+a12×Ta62→out1−out2*(T1/T2)=a161×Ta61−a162×Ta62*(T1/T2)  (2)

Here, adapting T1 and T2 to be the same as each other enablescalculation to be remarkably simplified. Moreover, adapting Ta61 andTa62 to be the same as each other enables to detect only a difference inreflectance.

Incidentally, with respect to computation processing after the output inthe case of FIG. 14, the computation processing is basically similar tothat in the example of FIG. 3 in which computation processing after theoutput in the case of FIG. 2 is described. Therefore, the descriptionthereof will be omitted.

Light Exposure Control of Pixels

FIG. 15 is a diagram illustrating the light emission timing of two kindsof active light and the light exposure timing of three kinds of pixels.It should be noted that in the case of this FIG. 15 as well, the lightexposure end timings in any respective two of the pixels 2-1 to 2-3 maybe the same as each other.

In a case where detection is performed more flexibly than the pixellight-exposure control described with reference to FIG. 14, on theassumption that outputs of an image 2-1, the pixel 2-2, and the pixel2-3 are out1, out2, and out3 respectively, intensity of reflected lightby the active light 61 is a161, intensity of reflected light by theactive light 62 is a162, intensity of background light is bg, as shownin FIG. 15, the light exposure time of the pixel 2-1 is T1, the lightexposure time of the pixel 2-2 is T2, the light exposure time of thepixel 2-3 is T3, and the times during which the lights are included inthe pixels 2-N are Ta(N-61) and Ta(N-62), the following equation (3) isobtained.

[Equation 3]

out1=bg×T1+a161×Ta(1-61)+a162×Ta(1-62)out2=bg×T2+a161×Ta(2-61)+a162×Ta(2-62)out3=bg×T3+a161×Ta(3-61)+a162×Ta(3-62)  (3)

Under a condition in which three equations in the above-describedequation (3) are not in a proportional relationship, unknown signals,bg, a161, and a162, can be calculated.

FIG. 16 is a diagram illustrating computation processing after output.

In an example of FIG. 16, the light exposure times (the timings) of thepixel 2-1, the pixel 2-2, and the pixel 2-3 among pixels of the pixelarea 3 are controlled. Although three pixels may be arbitrarilyselected, the three pixels are arranged as indicated in, for example, anoutput image 71. Pixels may be configured in units of 2×2, or variouspatterns, such as in units of 3×3, in units of 4×4, and in units of 2×4,can be considered.

As shown in an image 72-1, an image 72-2, and an image 72-3, this outputimage 71 is subjected to de-mosaic processing, and a computation unit 73performs computation processing on a pixel basis, thereby enabling alight exposure image 74-1 that is exposed to active light 1 and a lightexposure image 74-2 that is exposed to active light 2 to be obtained,the influence of background light being eliminated from the lightexposure image 74-1 and the light exposure image 74-2.

Similarly, in three-pixel driving, the driving described above withreference to FIG. 15 also enables detection that uses three lightsources. Moreover, by independently controlling the light exposuretimings of four pixels, it is also easy to increase the number of lightsources. By independently controlling N or N+1 pixels for N lightsources in sequence, the influence of background light can besubtracted.

In the case of a circuit configuration in which the light exposure timecan be completely independently controlled, the operations of FIGS. 13and 14 can be easily realized. However, there is, for example, a casewhere a transparent electrode cannot be independently controlled. Next,this case will be described with reference to FIG. 17.

Circuit Configuration and the Light Exposure Control Timing in the CaseOf Global Shutter

FIG. 17 shows a circuit configuration and the light exposure controltiming in a case where a transparent electrode cannot be independentlycontrolled. A of FIG. 17 shows an example of a circuit configuration ina case where a circuit is shared by three pixels.

In other words, the pixel 2-1, the pixel 2-2, and the pixel 2-3 eachinclude the photoelectric conversion film 41, the transparent electrode42, the lower electrode 43, and the TRG transistor 33. However, the RSTtransistor 36, the AMP transistor 37, the SEL transistor 38, and the FD39 are shared by the pixel 2-1 and the pixel 2-2. It should be notedthat in the case of A of FIG. 17, although the transparent electrode 42is illustrated in the same terminal as the VC, the transparent electrode42 may be physically separated.

As shown in B of FIG. 17, in the pixel 2-1, in the timing of inputtingof the TRG1 signal after a signal is input into the transparentelectrode 42 from the VC, light exposure is started before lightemission of the active light 61, and the light exposure is ended in thetiming in which the inputting of the VC is ended.

In the pixel 2-2, after light emission of the active light 61, and inthe timing of inputting of the TRG2 signal before light emission of theactive light 62, light exposure is started, and the light exposure isended in the timing in which inputting of the VC is ended. In the pixel2-3, after light emission of the active light 62, and in the timing ofinputting of the TRG3 signal before inputting of the VC is ended, lightexposure is started, and the light exposure is ended in the timing inwhich the inputting of the VC is ended.

Incidentally, according to the above-described light exposure control,on the assumption that outputs of the pixel 2-1, the pixel 2-2, and thepixel 2-3 are out1, out2, and out3 respectively, intensity of reflectedlight by the active light 61 is a161, intensity of reflected light bythe active light 62 is a162, intensity of background light is bg, thelight exposure time of the pixel 2-1 is T1, the light exposure time ofthe pixel 2-2 is T2, the light exposure time of the pixel 2-3 is T3, thelight emission time of the active light 61 is Ta61, and the lightemission time of the active light 62 is Ta62, the following equation (4)is obtained.

[Equation 4]

out1=bg×T1+a161×Ta61+a162×Ta62 out2=bg×T2+a162×Ta62 out3=bg×T3  (4)

The following equation (5) can be easily calculated from this equation(4).

[Equation 5]

a162×Ta62=out2−out3×(T2/T3) a161×Ta61=out1−out2−out3×(T1−T2)/T3  (5)

It should be noted that in a case where the circuit configuration shownin A of FIG. 17 is applied to 2×2 square arrangement pixels, oneremaining pixel among four pixels may be controlled in a manner similarto that of out3. In this case, TRG3 and 4 are driven in the same manner,and in a case where FD addition is possible, the addition is made. Thesignal amount of out3, the signal amount of which is small, increases,and consequently SN can be improved. In a case where FD addition is notpossible, even source follower addition or addition in a digital areaenables SN to be improved.

It should be noted that in the above explanation, as shown in A of FIG.18, a checker pattern of the pixel 2-1 and the pixel 2-2, and ablack-and-white sensor, are taken into consideration. However,application to other patterns and various arrays of color filters canalso be considered.

For example, as shown in B of FIG. 18, there can be considered anexample in which light exposure is controlled on a row basis, forexample, in the order of the row of the pixel 2-1, the row of the pixel2-2, the row of the pixel 2-1, and the row of the pixel 2-2, or anexample in which light exposure is controlled on a column basis, forexample, in the order of the column of the pixel 2-1, the column of thepixel 2-2, the column of the pixel 2-1, and the column of the pixel 2-2.

It should be noted that in a case where light exposure is controlled ona row basis as shown on the left side of B of FIG. 18, the control canbe performed without increasing a control line.

In addition, for example, as shown in C of FIG. 18, an example in whichcontrol is performed by using a 2×2 pattern can also be considered. Itshould be noted that light exposure may be controlled in units of, forexample, 2×2, 2×4, 4×2, 3×3, 4×4, or the like.

For example, as shown in A of FIG. 19, for Bayer pattern of 2×2 pixels,the light exposure time can also be controlled in units of 4×4 pixels.

Moreover, as shown in B of FIG. 19, for Bayer pattern in which 2×2pixels share the same color filter, the light exposure time can also becontrolled in the 2×2 pixels.

In addition, as another pixel pattern, as shown in A of FIG. 20, afilter pattern of 2×2 pixels is used in a color filter array having R,G, B, and W, and the present technology can be applied by changing thelight exposure time of a pixel at a W array point.

Moreover, as shown in B of FIG. 20, in a case where light exposure iscontrolled by using red active light, light exposure of only a pixelcorresponding to a red filter has only to be controlled. Light exposureof only a pixel corresponding to a color filter, the color of whichagrees with a color of active light, can also be controlled.

Incidentally, besides the above, there are various pixel arrays such asa 3×3 color pattern, and a 4×4, 5×5, or 6×6 pixel pattern. However, aswith the Bayer pattern shown in A of FIG. 19, for example, in the caseof a color filter in which 6×6 pixel units are repeated, the colorfilter increasing randomness of a color filter array to reduce aninfluence such as moire, light exposure may be controlled in units of6×6 pixels (in other words, the light exposure time of 6×6 pixels, andthe light exposure time of the next 6×6 pixels, are controlled), or thecontrol can be achieved even by controlling light exposure of the samecolor filter pixel included in 6×6 pixels.

Physical image of control lines FIGS. 21 to 23 are diagrams eachillustrating an example of a physical image of control lines.

FIG. 21 illustrates, as an example, an image of control lines in thecase of one pixel one read circuit (no pixel sharing). In addition, thisis an image of a black-and-white array.

In the image pickup device 1, light exposure control signal lines101L[0] to 101L[N], light exposure control signal lines 101R[0] to101R[0], and the other control signal lines 102[0] to[N] are wired tothe pixel 2 in the pixel area 3.

The light exposure control signal lines 101L[0] to 101L[N] and the lightexposure control signal lines 101R[0] to 101R[0] indicate wiring linesfor controlling the light exposure time, for example, the TRG signal,the TRX signal, the OFG signal, the RST signal, or the like. Althoughillustrated with one line here, in a case where the light exposure timeis controlled by combining the OFG signal with the TRX signal, two linesare arranged.

The other control signal lines [0] to [N] become control lines forreading signals from circuits such as the SEL transistor 38 and the RSTtransistor 36. The other control signal lines [0] to [N] become signalsother than the light exposure control lines among all control lines.

A signal from each pixel is input, through a vertical signal line 9,into a comparator 121 that constitutes a part of an Analog/Digital (A/D)conversion circuit of the column signal processing circuit 5 of FIG. 1.The signal is then subjected to predetermined signal processing. Thesignal is input into, for example, the comparator 121 that constitutes apart of the A/D conversion circuit. The comparator 121 compares theinput signal with a value of a DAC 112. A clock from a clock generator111 that constitutes a part of the control circuit 8 is input into thecomparator 121 by the DAC 112 as analog data, and is directly input intoa counter 122.

It should be noted that RST or the like also becomes a light exposurecontrol signal, or can also become any of other control signals. Thenumber of wiring lines varies from 1 to 5 according to a control method.

In a case where more complicated control is performed, for example, whenthree or more pixels that differ in the light exposure timing areprepared, the number of light exposure control lines has only to beincreased.

FIG. 22 illustrates, as an example, an image of control lines used in acase where two upper and lower pixels are shared. In addition, this isalso an image of a black-and-white array.

FIG. 23 shows an example in which the present technology is applied toBayer array. The light exposure control signal lines 101-1[0] to101-3[0] are connected to upper pixels among four shared pixels, and thelight exposure control signal lines 101-4[0] to 101-4[0] are connectedto lower pixels.

In the case of the example of FIG. 23, the light exposure time can beindependently controlled by two R and B in the same row, and a G pixelthat is away therefrom by two rows.

It should be noted that the above description is merely an example, andthe present technology can be applied to various color filter patterns.By the number of pixels for which the light exposure time is desired tobe controlled, the number of light exposure control wiring lines hasonly to be supported corresponding to the block.

3. Third Embodiment Other Configuration Examples

As other configurations, the present technology can also be applied tothe image pickup device 150 such as that shown in FIG. 24, the imagepickup device 150 using an area ADC method in which ADC is arranged forevery two or more areas in both x and Y. It is possible to conceive of,for example, a configuration in which a pixel substrate 151 on which aplurality of pixel units 171 each having a plurality of pixel circuits161 arranged therein are arranged, and a circuit substrate 152 on whicha plurality of circuit blocks 181 corresponding to the pixel units 171are arranged, are laminated together by micro-bump, TSV, Cu-Cu joining,or the like. The present technology can be applied to the configuration.

In addition, with respect to the pixel ADC method in which one ADC isarranged in one pixel, changing the light exposure time of each pixelenables similar support to be performed.

Moreover, as shown in FIG. 25, for example, a pixel circuit thatincludes the TRG transistor 33, the PD 34, the RST transistor 36, theAMP transistor 37, the SEL transistor 38, and the FD 39 may be combinedwith a conversion efficiency adjustable function 200 that includes atransistor 201 and a capacitor 202.

In this case, a conversion efficiency on the side where a signal levelis low (although this is the side where the light exposure time isshort, it is not always so depending on an input ratio of active light)can be increased. As the result, a signal level on the side where thesignal level is low can be increased, and SN can be improved.

4. Example of Using Image Sensor

FIG. 26 is a diagram illustrating an example of using theabove-described solid-state image pickup device.

The above-described solid-state image pickup device (image sensor) canbe used for various cases of sensing light such as visible light,infrared light, ultraviolet light, and X-ray, for example, as describedbelow.

-   -   A device for taking an image for appreciation, the device        including a digital camera, a portable apparatus having a camera        function, and the like    -   A device for traffic purposes, the device including: for        example, a vehicle-mounted sensor for imaging, for example, the        front and rear of an automobile, and the surrounding and inside        of the automobile, for the purposes of, for example, safe        driving such as automatic stop, and the recognition of a state        of a driver; a monitoring camera for monitoring traveling        vehicles and roads; and a distance measuring sensor for        measuring, for example, a distance between vehicles    -   A device for imaging a gesture of a user to enable apparatus        operation according to the gesture to be performed, the device        being used in a home electric appliance such as a TV, a        refrigerator, and an air conditioner    -   A device used for medical care or health care, the device        including an endoscope, a device for taking an image of blood        vessels by receiving infrared light, and the like    -   A device for security purposes, the device including a        monitoring camera for security use, a camera used for person        authentication, and the like    -   A device for beauty purposes, the device including a skin        measuring instrument for imaging skin, a microscope for imaging        a scalp, and the like    -   A device for sports purposes, the device including for example,        a wearable camera and an action camera that are used for sports        or the like    -   A device for agricultural purposes, the device including a        camera for monitoring a state of a field and a crop, and the        like

5. Example of Electronic Apparatus Configuration Example of ElectronicApparatus

Moreover, the application of the present technology is not limited tothe application to the solid-state image pickup device. The presenttechnology can also be applied to an image pickup device. Here, theimage pickup device includes: a camera system such as a digital stillcamera and a digital video camera; and an electronic apparatus having animage pickup function, such as a portable telephone. It should be notedthat there is also a case where a module-like form provided in anelectronic apparatus, that is to say, a camera module, is treated as animage pickup device.

Here, a configuration example of an electronic apparatus according tothe present technology will be described with reference to FIG. 27.

An electronic apparatus 300 shown in FIG. 27 is provided with asolid-state image pickup device (element chip) 301, an optical lens 302,a shutter device 303, a driving circuit 304, and a signal processingcircuit 305. As the solid-state image pickup device 301, the solid-stateimage pickup device 1 according to the present technology describedabove is provided. In addition, the electronic apparatus 300 is providedwith a light-emitting unit of the above-described active light as anunillustrated light-emitting unit. It should be noted that as the signalprocessing path 505, the computation unit 23 of FIG. 3 and thecomputation unit 73 of FIG. 16 are provided.

The optical lens 302 forms an image of image light (incident light) froman object on an image pickup surface of the solid-state image pickupdevice 301. As the result, a signal charge is accumulated in thesolid-state image pickup device 301 for a fixed period of time. Theshutter device 303 controls a light irradiation time period and a lightshielding time period for the solid-state image pickup device 301.

The driving circuit 304 supplies driving signals for controlling thesignal transfer operation of the solid-state image pickup device 301,the shutter operation of the shutter device 303, and the light-emittingoperation of the unillustrated light-emitting unit. The driving circuit304 controls each operation by using parameters set by an unillustratedCPU. The solid-state image pickup device 301 transfers a signal by thedriving signal (timing signal) supplied from the driving circuit 304.The signal processing circuit 305 subjects the signal output from thesolid-state image pickup device 301 to various kinds of signalprocessing. A video signal that has been subjected to the signalprocessing is stored in a storage medium such as a memory, or is outputto a monitor.

6. Example of Application to Endoscopic Operation System

The technology (the present technology) according to the presentdisclosure can be applied to various products. For example, thetechnology according to the present disclosure may be applied to anendoscopic operation system.

FIG. 28 is a diagram illustrating an example of a schematicconfiguration of an endoscopic operation system to which the technology(the present technology) according to the present disclosure can beapplied.

FIG. 28 illustrates a state in which an operator (doctor) 11131 uses anendoscopic operation system 11000 to carry out an operation on a patient11132 on a patient bed 11133. As Illustrated in the figure, theendoscopic operation system 11000 includes an endoscope 11100, othersurgical tools 11110 such as an aeroperitoneum tube 11111 and an energytreatment tool 11112, a support arm device 11120 that supports theendoscope 11100, and a cart 11200 equipped with various devices forendoscopic surgery.

The endoscope 11100 includes a lens tube 11101 having an area that has apredetermined length from the end, and that is inserted into a bodycavity of the patient 11132, and a camera head 11102 that is connectedto a base end of the lens tube 11101. In the example shown in thefigure, the endoscope 11100 that is configured as what is called a hardmirror having a hard lens tube 11101 is illustrated. However, theendoscope 11100 may be configured as what is called a soft mirror havinga soft lens tube.

The end of the lens tube 11101 is provided with an opening into which anobjective lens is fitted. A light source device 11203 is connected tothe endoscope 11100. Light generated by the light source device 11203 isguided to the end of the lens tube by a light guide extended inside thelens tube 11101, and is emitted, through the objective lens, toward anobservation target in the body cavity of the patient 11132. It should benoted that the endoscope 11100 may be a direct view mirror, or may be anoblique view mirror or a side view mirror.

An optical system and an image pickup element are provided inside thecamera head 11102, and reflected light (observation light) from theobservation target is concentrated into the image pickup element by theoptical system. The observation light is photoelectrically converted bythe image pickup element, and an electric signal corresponding to theobservation light, in other words, an image signal corresponding to anobservation image, is generated. The image signal is transmitted to acamera control unit (Camera Control Unit (CCU)) 11201 as RAW data.

The CCU 11201 includes a Central Processing Unit (CPU), a GraphicsProcessing Unit (GPU), and the like. The CCU 11201 controls theoperation of the endoscope 11100 and the operation of the display device11202 in a unified manner. Moreover, the CCU 11201 receives an imagesignal from the camera head 11102, and subjects the image signal tovarious image processing for displaying an image based on the imagesignal, for example, development processing (de-mosaic processing) orthe like.

By the control from the CCU 11201, the display device 11202 displays animage based on the image signal that has been subjected to the imageprocessing by the CCU 11201.

The light source device 11203 includes, for example, a light source suchas a light emitting diode (LED), and supplies the endoscope 11100 withirradiation light used when an operated region or the like is imaged.

An input device 11204 is an input interface with the endoscopicoperation system 11000. A user is allowed to input various informationor an instruction into the endoscopic operation system 11000 through theinput device 11204. For example, the user inputs an instruction or thelike to change image pickup conditions (a kind of irradiation light, amagnification ratio, a focal length, etc.) of image pickup by theendoscope 11100.

A treatment tool control device 11205 controls driving of the energytreatment tool 11112 that is used for, for example, cauterizing orincising a tissue, or sealing a blood vessel. For the purpose ofsecuring a visual field by the endoscope 11100, and for the purpose ofsecuring a work space of an operator, an aeroperitoneum device 11206sends a gas into the body cavity through the aeroperitoneum tube 11111so as to cause the body cavity of the patient 11132 to swell. A recorder11207 is a device that is capable of recording various informationrelated to an operation. A printer 11208 is a device that is capable ofprinting various information related to an operation in various formatssuch as a text, an image, or a graph.

Incidentally, the light source device 11203 that supplies irradiationlight used when an operated region is imaged by the endoscope 11100 canbe configured from a white light source that is configured by, forexample, an LED, a laser light source, or a combination thereof. In acase where the white light source is configured by a combination of RGBlaser light sources, the output strength and output timing of each color(each wavelength) can be controlled with high accuracy. Therefore, whitebalance of a picked-up image can be adjusted in the light source device11203. In addition, in this case, by irradiating the observation targetwith a laser beam from each of the RGB laser light sources by timedivision, and by controlling driving of an image pickup element of thecamera head 11102 in synchronization with the irradiation timing, animage corresponding to each RGB can also be picked up by time division.According to the method, even if the image pickup element is notprovided with a color filter, a color image can be obtained.

Further, driving of the light source device 11203 may be controlled insuch a manner that light intensity of output light is changed at everypredetermined time. By controlling driving of the image pickup elementof the camera head 11102 in synchronization with the timing of changingthe light intensity to obtain images by time division, and bysynthesizing the images, an image having a high dynamic range, which isfree from what is called blocked-up shadows and blown-out highlights,can be generated.

Moreover, the light source device 11203 may be configured so as to becapable of supplying light in a predetermined wavelength bandcorresponding to special light observation. In the special lightobservation, by using, for example, wavelength dependence of absorptionof light in a body tissue, narrow-band light in comparison withirradiation light (that is to say, white light) at the time of ordinaryobservation is emitted to image a predetermined tissue such as a bloodvessel of a mucous membrane surface layer with high contrast. What iscalled, narrow-band light observation (Narrow Band Imaging) isperformed. Alternatively, in the special light observation, fluorescentobservation that obtains an image by fluorescence generated by beingirradiated with excitation light may be performed. In the fluorescentobservation, for example, fluorescence from a body tissue can beobserved (autofluorescence observation) by irradiating the body tissuewith excitation light, or a fluorescent image can be obtained by locallyinjecting a reagent such as indocyanine green (ICG) into a body tissue,and by irradiating the body tissue with excitation light correspondingto a fluorescent wavelength of the reagent. The light source device11203 can be configured to be capable of supplying narrow-band lightand/or excitation light corresponding to such special light observation.

FIG. 29 is a block diagram illustrating an example of a functionalconfiguration of the camera head 11102 and the CCU 11201 shown in FIG.28.

The camera head 11102 includes a lens unit 11401, an image pickup unit11402, a drive unit 11403, a communication unit 11404, and a camera headcontrol unit 11405. The CCU 11201 includes a communication unit 11411,an image processing unit 11412, and a control unit 11413. The camerahead 11102 and the CCU 11201 are connected by a transmission cable 11400so as to be capable of communicating with each other.

The lens unit 11401 is an optical system that is provided in aconnection unit for connecting to the lens tube 11101. Observation lighttaken from the end of the lens tube 11101 is guided to the camera head11102, and enters the lens unit 11401. The lens unit 11401 is configuredby combining a plurality of lenses including a zoom lens and a focusinglens.

The number of image pickup elements that constitute the image pickupunit 11402 may be one (what is called, single plate) or two or more(what is called, multi-plate). In a case where the image pickup unit11402 is configured by multi-plate, a color image may be obtained bygenerating an image signal corresponding to each RGB by, for example,each image pickup element, and by synthesizing the image signals.Alternatively, the image pickup unit 11402 may be configured so as tohave a pair of image pickup elements for obtaining image signals forright and left eyes respectively, the image signals supporting 3D(dimensional) display. Performing three dimensional display enables theoperator 11131 to more correctly grasp a depth of a biological tissue inan operated region. It should be noted that in a case where the imagepickup unit 11402 is configured by multi-plate, the number of lens units11401 that can be provided is also two or more in response to each imagepickup element.

In addition, it is not always necessary to provide the image pickup unit11402 in the camera head 11102. The image pickup unit 11402 may beprovided, for example, inside the lens tube 11101, and immediately afterthe objective lens.

The drive unit 11403 is configured by an actuator. By the control fromthe camera head control unit 11405, the drive unit 11403 moves the zoomlens and focusing lens of the lens unit 11401 only by a predetermineddistance along an optical axis. This enables the magnification ratio andfocus of a picked-up image obtained by the image pickup unit 11402 to beadjusted, as appropriate.

The communication unit 11404 is configured by a communication device fortransmitting and receiving various information to and from the CCU11201. The communication unit 11404 transmits the image signal obtainedfrom the image pickup unit 11402 to the CCU 11201 through thetransmission cable 11400 as RAW data.

In addition, the communication unit 11404 receives a control signal forcontrolling driving of the camera head 11102 from the CCU 11201, andsupplies the control signal to the camera head control unit 11405. Thecontrol signal includes, for example, information related to imagepickup conditions such as: information that a frame rate of a picked-upimage is specified; information that an exposure value at the time ofimage pickup is specified; and/or information that a magnification ratioof the picked-up image and a focus are specified.

It should be noted that the image pickup conditions such as the framerate, the exposure value, the magnification ratio, and the focus may bespecified as appropriate by a user, or may be automatically set by thecontrol unit 11413 of the CCU 11201 on the basis of the obtained imagesignal. In the latter case, the endoscope 11100 is equipped with, whatis called, an Auto Exposure (AE) function, an Auto Focus (AF) function,and an Auto White Balance (AWB) function.

The camera head control unit 11405 controls driving of the camera head11102 on the basis of the control signal received from the CCU 11201through the communication unit 11404.

The communication unit 11411 is configured by a communication deviceused to transmit/receive various information to/from the camera head11102. From the camera head 11102, the communication unit 11411 receivesan image signal transmitted through the transmission cable 11400.

In addition, the communication unit 11411 transmits, to the camera head11102, a control signal for controlling driving of the camera head11102. The image signal and the control signal can be transmitted byelectrical communication, optical communication, or the like.

The image processing unit 11412 subjects the image signal that has beentransmitted from the camera head 11102, and that is RAW data, to variousimage processing.

The control unit 11413 carries out the various control related to imagepickup of an operated region or the like by the endoscope 11100, andrelated to displaying of a picked-up image obtained by image pickup ofan operated region or the like. For example, the control unit 11413generates a control signal for controlling driving of the camera head11102.

Moreover, the control unit 11413 causes the display device 11202 todisplay the picked-up image containing the operated region or the likeon the basis of the image signal that has been subjected to the imageprocessing by the image processing unit 11412. In this case, the controlunit 11413 may recognize various objects contained in the picked-upimage by using various image recognition technologies. For example, bydetecting a shape, a color, or the like of the edge of an objectincluded in the picked-up image, the control unit 11413 is capable ofrecognizing, for example, a surgical tool such as forceps, a specificbiological region, bleeding, or a mist at the time of using the energytreatment tool 11112. When the control unit 11413 causes the displaydevice 11202 to display the picked-up image, the control unit 11413 maycause various operation support information to be displayed so as to besuperimposed on an image of the operated region by using the recognitionresult. By displaying the operation support information in asuperimposed manner to present the operation support information to theoperator 11131, a load of the operator 11131 can be reduced, and theoperator 11131 is enabled to make an operation progress reliably.

The transmission cable 11400 that connects between the camera head 11102and the CCU 11201 is an electric signal cable corresponding tocommunication of an electric signal, an optical fiber corresponding tooptical communication, or a composite cable thereof.

Here, in the example shown in the figure, the communication is wiredlyperformed by using the transmission cable 11400. However, thecommunication between the camera head 11102 and the CCU 11201 may bewirelessly performed.

The example of the endoscopic operation system to which the technologyaccording to the present disclosure can be applied has been explainedabove. The technology according to the present disclosure can be appliedto, for example, the endoscope 11100, (the image pickup unit 11402 of)the camera head 11102, the image processing unit 11412 of the CCU 11201,the light source device 11203, and the like, among the configurationsdescribed above. Specifically, for example, the solid-state image pickupdevice 1 of FIG. 1 can be applied to the image pickup unit 11402. Forexample, the computation unit 23 of FIG. 3 and the computation unit 73of FIG. 16 can be applied to the image processing unit 11412. Theunillustrated light-emitting unit of active light can be applied to thelight source device 11203. By applying the technology according to thepresent disclosure to the image pickup unit 11402 and the imageprocessing unit 11412, a clearer operated region image can be obtained,which enables an operator to reliably check an operated region. Further,as described above with reference to FIG. 14, what is desired to bedetected can be detected from a difference in absorption coefficient byusing light having two wavelengths, and therefore, a change inconcentration or the like of, for example, oxyhemoglobin ordeoxyhemoglobin can also be calculated.

It should be noted that although the endoscopic operation system hasbeen explained as an example here, the technology according to thepresent disclosure may be applied to, for example, a microscopicoperation system or the like besides the above.

7. Example of Application to Moving Object

The technology (the present technology) according to the presentdisclosure can be applied to various products. For example, thetechnology according to the present disclosure may be realized as adevice that is provided in any kind of moving objects including anautomobile, an electric vehicle, a hybrid electric vehicle, amotorcycle, a bicycle, a personal mobility, an airplane, a drone, aship, a robot, and the like.

FIG. 30 is a block diagram illustrating an example of a schematicconfiguration of a vehicle control system that is an example of a movingobject control system to which the technology according to the presentdisclosure can be applied.

A vehicle control system 12000 is provided with a plurality ofelectronic control units that are connected through the communicationnetwork 12001. In the example shown in FIG. 30, the vehicle controlsystem 12000 is provided with a drive system control unit 12010, a bodysystem control unit 12020, a vehicle outside information detection unit12030, a vehicle inside information detection unit 12040, and anintegrated control unit 12050. In addition, a microcomputer 12051, asound-image output unit 12052, and an in-vehicle network Interface (I/F)12053 are illustrated as a functional configuration of the integratedcontrol unit 12050.

The drive system control unit 12010 controls the operation of a devicerelated to a drive system of a vehicle according to various kinds ofprograms. For example, the drive system control unit 12010 functions asa control device for: a driving force generator that generates thedriving force of the vehicle, such as an internal combustion engine or adriving motor; a driving force transmission mechanism for transferringthe driving force to a wheel; a steering mechanism for adjusting arudder angle of the vehicle; a braking device that generates the brakingforce of the vehicle; and the like.

The body system control unit 12020 controls the operation of variouskinds of devices provided in the vehicle body according to various kindsof programs. For example, the body system control unit 12020 functionsas a control device for a keyless entry system, a smart key system, apower window device, or various kinds of lamps such as a head lamp, aback lamp, a brake lamp, a blinker, and a fog lamp. In this case, anelectrical wave transmitted from a portable device that is substitutedfor a key, or signals of various switches, can be input into the bodysystem control unit 12020. The body system control unit 12020 acceptsthe input of the electrical wave or the signals, and controls a doorlock device, a power window device, a lamp, and the like of the vehicle.

The vehicle outside information detection unit 12030 detects informationof the outside of the vehicle equipped with the vehicle control system12000. For example, the image pickup unit 12031 is connected to thevehicle outside information detection unit 12030. The vehicle outsideinformation detection unit 12030 causes the image pickup unit 12031 topick up an image outside the vehicle, and receives the picked-up image.The vehicle outside information detection unit 12030 may perform objectdetection processing or distance detection processing of a person, acar, an obstacle, a sign, characters on a road surface, or the like onthe basis of the received image.

The image pickup unit 12031 is an optical sensor that receives light tooutput an electric signal corresponding to the received amount of thelight. The image pickup unit 12031 is also capable of outputting anelectric signal as an image, and is also capable of outputting theelectric signal as information of distance measurement. In addition,light received by the image pickup unit 12031 may be visible light, ormay be non-visible light such as infrared rays.

The vehicle inside information detection unit 12040 detects informationinside the vehicle. For example, a driver state detection unit 12041that detects a state of a driver is connected to the vehicle insideinformation detection unit 12040. The driver state detection unit 12041includes, for example, a camera for picking up an image of the driver.The vehicle inside information detection unit 12040 may calculate afatigue degree, or a concentration degree, of the driver, or maydetermine whether or not the driver is dozing, on the basis of detectioninformation input from the driver state detection unit 12041.

The microcomputer 12051 is capable of computing a control target valueof the driving force generator, the steering mechanism, or the brakingdevice on the basis of information inside or outside the vehicle, whichis obtained by the vehicle inside information detection unit 12040 orthe vehicle outside information detection unit 12030, and is capable ofoutputting a control instruction to the drive system control unit 12010.For example, the microcomputer 12051 is capable of performingcooperative control for the purpose of realizing a function of anAdvanced Driver Assistance System (ADAS) including the vehicle collisionavoidance or shock mitigation, follow-up traveling based on a distancebetween vehicles, vehicle-speed maintaining traveling, a vehiclecollision warning, a vehicle lane deviation warning, or the like.

In addition, by controlling the driving force generator, the steeringmechanism, the braking device, or the like on the basis of informationaround the vehicle, which is obtained by the vehicle outside informationdetection unit 12030 or the vehicle inside information detection unit12040, the microcomputer 12051 is capable of performing cooperativecontrol for the purpose of, for example, automatic driving that causes avehicle to autonomously travel without depending on driver's operation.

Moreover, the microcomputer 12051 is capable of outputting a controlinstruction to the body system control unit 12030 on the basis ofinformation outside the vehicle obtained by the vehicle outsideinformation detection unit 12030. For example, the microcomputer 12051controls a head lamp according to a position of a preceding vehicle oran oncoming vehicle, which is detected by the vehicle outsideinformation detection unit 12030, thereby enabling cooperative controlfor the purpose of taking anti-glare measures, for example, switchinghigh-beam to low-beam to be performed.

The sound-image output unit 12052 transmits at least one of an audiooutput signal or an image output signal to an output device that iscapable of visually or audibly notifying passengers of the vehicle orpersons outside the vehicle of information. In the example in FIG. 30,an audio speaker 12061, a display unit 12062, and an instrument panel12063 are presented as output devices. The display unit 12062 mayinclude at least one of, for example, an on-board display or a head-updisplay.

FIG. 31 is a diagram illustrating an example of a position at which theimage pickup unit 12031 is provided.

In FIG. 31, image pickup units 12101, 12102, 12103, 12104, and 12105 areprovided as image pickup units 12031.

The image pickup units 12101, 12102, 12103, 12104, and 12105 areprovided at positions of, for example, a front nose, a side-view mirror,a rear bumper, and a back door of the vehicle 12100, and at a positionof, for example, an upper part of a windshield inside the vehicle room.The image pick up unit 12101 provided at the front nose, and the imagepick up unit 12105 provided at the upper part of the windshield insidethe vehicle room, mainly obtain an image viewed from the front of thevehicle 12100. The image pick up units 12102 and 12103 provided at theside-view mirrors respectively mainly obtain images viewed from thesides of the vehicle 12100. The image pick up unit 12104 provided at therear bumper or the back door mainly obtains an image viewed from theback of the vehicle 12100. The image pick up unit 12105 provided at theupper part of the windshield inside the vehicle room is mainly used todetect preceding vehicles, or walkers, obstacles, traffic lights,traffic signs, traffic lanes, or the like.

It should be noted that FIG. 31 shows, as an example, imaging ranges ofthe respective image pickup units 12101 to 12104. An image pick up range12111 indicates an image pick up range of the image pick up unit 12101provided at the front nose; image pick up ranges 12112 and 12113indicate image pick up ranges of the image pick up units 12102 and 12103provided at the side-view mirrors respectively; and an image pick uprange 12114 indicates an image pick up range of the image pick up unit12104 provided at the rear bumper or the back door. Superimposing imagedata picked up by, for example, the image pick up units 12101 to 12104enables a bird's-eye view image of the vehicle 12100 viewed from theupper part to be obtained.

At least one of the image pickup units 12101 to 12104 may have afunction of obtaining distance information. For example, at least one ofthe image pickup units 12101 to 12104 may be a stereo camera thatincludes a plurality of image pickup elements, or may be an image pickupelement having a pixel for detecting a phase difference.

For example, the microcomputer 12051 determines a distance to eachthree-dimensional object in the image capturing ranges 12111 to 12114,and a temporal change of this distance (a relative speed with respect tothe vehicle 12100), on the basis of distance information obtained fromthe image pickup units 12101 to 12104. Consequently, in particular, athree-dimensional object that is the closest on a traveling path of thevehicle 12100, and that is traveling at a predetermined speed (forexample, 0 km/h or more) substantially in the same direction as that ofthe vehicle 12100, can be extracted as a preceding vehicle. Moreover,the microcomputer 12051 sets a distance between vehicles, which shouldbe kept beforehand behind a preceding vehicle. Consequently, automaticbrake control (also including follow-up stop control), automaticacceleration control (also including follow-up start control), and thelike can be performed. In this manner, the microcomputer 12051 iscapable of performing cooperative control for the purpose of, forexample, automatic driving that causes a vehicle to autonomously travelwithout depending on driver's operation.

For example, on the basis of distance information obtained from theimage pickup units 12101 to 12104, the microcomputer 12051 classifiesthree-dimensional object data related to three-dimensional objects intoother three-dimensional objects such as a two-wheeled vehicle, anordinary vehicle, a large-sided vehicle, a walker, and a utility pole,and then extracts the three-dimensional object data. The microcomputer12051 is capable of using the three-dimensional object data toautomatically avoid obstacles. For example, the microcomputer 12051identifies an obstacle around the vehicle 12100 as an obstacle that canbe visually recognized by a driver of the vehicle 12100 or as anobstacle that is hard to visually recognize. In addition, themicrocomputer 12051 determines a collision risk indicating a degree ofrisk of collision with each obstacle. When the collision risk is a setvalue or higher, and thus is in a situation in which there is apossibility of collision, driving assistance for avoiding a collisioncan be performed by outputting a warning to a driver through the audiospeaker 12061 or the display unit 12062, or by carrying out forceddeceleration and avoidance steering through the drive system controlunit 12010.

At least one of the image pickup units 12101 to 12104 may be an infraredcamera that detects infrared rays. For example, by determining whetheror not a walker exists in picked-up images of the image pickup units12101 to 12104, the microcomputer 12051 is capable of recognizing thewalker. Such recognition of a walker is performed by: a step ofextracting, for example, feature points in the picked-up images of theimage pickup units 12101 to 12104 that are infrared cameras; and a stepof subjecting a series of feature points indicating an outline of anobject to pattern matching processing, and determining whether or notthe object is a walker. If the microcomputer 12051 determines that awalker exists in the picked-up images of the image pickup units 12101 to12104, and consequently recognizes the walker, the sound-image outputunit 12052 controls the display unit 12062 in such a manner that arectangular profile line for emphasis is superimposed on the recognizedwalker. In addition, the sound-image output unit 12052 may control thedisplay unit 12062 in such a manner that an icon or the like indicatinga walker is displayed at a desired position.

The example of the vehicle control system to which the technologyaccording to the present disclosure can be applied has been explainedabove. The technology according to the present disclosure can be appliedto the image pickup unit 12031 (including the image pickup units 12101to 12104) among the configurations described above. Specifically, forexample, the solid-state image pickup device 1 of FIG. 1 can be appliedto the image pickup unit 12031. The computation unit 23 of FIG. 3 andthe computation unit 73 of FIG. 16 can be applied to the vehicle outsideinformation detection unit 12030 or the like. The unillustratedlight-emitting unit of active light can be applied to the body systemcontrol unit 12020. By applying the technology according to the presentdisclosure to the image pickup unit 12031, when background light isdesired to be eliminated in, for example, an on-vehicle device, exposuredoes not extend over frames. Therefore, a particular effect of beingstrong in movement can be achieved.

It should be noted that in the present description, the step that statesthe series of processing includes, of course, not only the processingthat is performed according to the above-described order in atime-series manner, but also processing that is executed in parallel orindividually even if the processing is not necessarily handled in atime-series manner.

In addition, the embodiments in the present disclosure are not limitedto the embodiments described above. Various modifications can be madewithin the scope that does not deviate from the gist of the presentdisclosure.

Additionally, the configuration that has been explained as above as onedevice (or processing unit) may be divided so as to be configured as aplurality of devices (or processing units). In contrast, theconfiguration that has been explained as above as the plurality ofdevices (or processing units) may be put together so as to be configuredas one device (or processing unit). Moreover, it is needless to say thata configuration other than those described above may be added to theconfiguration of each device (or each processing unit). Furthermore, ifthe configuration and operation as a system as a whole are substantiallythe same, a part of a configuration of a certain device (or processingunit) may be included in a configuration of another device (or anotherprocessing unit). In other words, the present technology is not limitedto the above-described embodiments, and various modifications can bemade within the scope that does not deviate from the gist of the presenttechnology.

The preferable embodiments of the present disclosure have been describedin detail as above with reference to the accompanying drawings. However,the disclosure is not limited to the above-described examples. It isclear that if a person has ordinary skill in the technical field towhich the present disclosure belongs, the person is able to conceive ofvarious correction examples or modification examples within the categoryof the technical idea set forth in the claims. It should be understoodthat, of course, these examples also belong to the technical range ofthe present disclosure.

It should be noted that the present technology can also employ thefollowing configurations.

(1) A solid-state image pickup device including:

a pixel array unit on which pixels are two-dimensionally arranged; and

a light exposure control unit that controls light exposure time of afirst pixel, and light exposure time of a second pixel that differs inlight exposure time from the first pixel in such a manner that at leastone of light exposure start time or light exposure end time differsaccording to light emission time of predetermined light in the pixelarray unit.

(2) The solid-state image pickup device set forth in the preceding (1),in which

the light exposure control unit controls the light exposure time of thefirst pixel and the light exposure time of the second pixel in such amanner that the light exposure start time differs according to the lightemission time of the predetermined light.

(3) The solid-state image pickup device set forth in the preceding (1)or (2), in which

the light exposure control unit controls the light exposure time of thefirst pixel and the light exposure time of the second pixel in such amanner that the light exposure start time differs by changing aninclusion ratio of the light emission time of the predetermined light.

(4) The solid-state image pickup device set forth in any of thepreceding (1) to (3), in which

the pixel includes a PD.

(5) The solid-state image pickup device set forth in the preceding (4),in which

the light exposure control unit controls the light exposure start timeby reset operation, and controls the light exposure end time by chargetransfer.

(6) The solid-state image pickup device set forth in any of thepreceding (1) to (3), in which

the pixel includes an organic or inorganic photoelectric conversionfilm.

(7) The solid-state image pickup device set forth in the preceding (6),in which

the light exposure control unit controls the light exposure start timeby reset operation, and controls the light exposure end time by an upperelectrode of the photoelectric conversion film.

(8) The solid-state image pickup device set forth in the preceding (6),in which

the light exposure control unit controls the light exposure start timeof at least one of the first pixel or the second pixel by an upperelectrode of the photoelectric conversion film, and controls the lightexposure end time by the upper electrode of the photoelectric conversionfilm.

(9) The solid-state image pickup device set forth in the preceding (1),in which

the light exposure control unit controls the light exposure time of thefirst pixel and the light exposure time of the second pixel in such amanner that at least one of the light exposure start time or the lightexposure end time differs according to light emission times of aplurality of the predetermined lights.

(10) The solid-state image pickup device set forth in the preceding (9),in which

the light exposure control unit controls the light exposure time of thefirst pixel and the light exposure time of the second pixel in such amanner that at least one of the light exposure start time or the lightexposure end time differs by changing inclusion ratios of the lightemission times of a plurality of the predetermined lights respectively.

(11) The solid-state image pickup device set forth in any of thepreceding (1) to (10), further including

a computation unit that subjects images from the first pixel and thesecond pixel to mosaic processing, and performs computation processingon a pixel basis.

(12) The solid-state image pickup device set forth in the preceding (1),in which

the light exposure control unit controls the light exposure time of thefirst pixel, the light exposure time of the second pixel, and the lightexposure time of a third pixel that differs in light exposure time fromthe first pixel and the second pixel in such a manner that at least oneof the light exposure start time or the light exposure end time differsaccording to light emission times of a plurality of the predeterminedlights.

(13) The solid-state image pickup device set forth in the preceding(12), in which

the light exposure control unit controls the light exposure time of thefirst pixel, the light exposure time of the second pixel, and the lightexposure time of the third pixel in such a manner that at least one ofthe light exposure start time or the light exposure end time differsaccording to the light emission times of the plurality of predeterminedlights.

(14) The solid-state image pickup device set forth in the preceding (12)or (13), further including

a computation unit that subjects images from the first pixel, the secondpixel and the third pixel to mosaic processing, and performs computationprocessing on a pixel basis.

(15) The solid-state image pickup device set forth in any of thepreceding (1) to (14), in which

the pixel array unit includes a pixel having a conversion efficiencyadjustable function.

(16) An image pickup method including the step of

controlling, by a solid-state image pickup device, light exposure timeof a first pixel, and light exposure time of a second pixel that differsin light exposure time from the first pixel in such a manner that atleast one of light exposure start time or light exposure end timediffers according to light emission time of predetermined light in apixel array unit on which pixels are two-dimensionally arranged.

(17) An electronic apparatus including:

a light-emitting unit that emits light; and

a solid-state image pickup device, the solid-state image pickup deviceincluding:

a pixel array unit on which pixels are two-dimensionally arranged; and

a light exposure control unit that controls light exposure time of afirst pixel, and light exposure time of a second pixel that differs inlight exposure time from the first pixel in such a manner that at leastone of light exposure start time or light exposure end time differsaccording to light emission time of light emitted by the light-emittingunit in the pixel array unit.

REFERENCE SIGNS LIST

-   1 Solid-state image pickup device-   2, 2-1 to 2-3 Pixel-   3 Pixel area-   5 Column signal processing circuit-   7 Vertical control line-   8 Control circuit-   21 Output image-   22-1, 22-2 Image-   23 Computation unit-   24 Light exposure image-   31 OFG transistor-   32 TRX transistor-   33 TRG transistor-   34 PD-   35 Capacitor-   36 RST transistor-   37 AMP transistor-   38 SEL transistor-   39 FD-   41 Photoelectric conversion film-   42 Transparent electrode-   43 Lower electrode-   61, 62 Active light-   71 Output image-   72-1, 72-2 Image-   73 Computation unit-   74 Light exposure image-   111 Clock generator-   112 DAC-   121 Comparator-   122 Counter-   150 Image pickup device-   151 Pixel substrate-   152 Circuit substrate-   161 Pixel circuit-   171 Pixel unit-   181 Circuit block-   200 Conversion efficiency adjustable function-   201 Transistor-   202 Capacitor-   300 Electronic apparatus-   301 Solid-state image pickup device-   302 Optical lens-   303 Shutter device-   304 Driving circuit-   305 Signal processing circuit

1. A solid-state image pickup device comprising: a pixel array unit onwhich pixels are two-dimensionally arranged; and a light exposurecontrol unit that controls light exposure time of a first pixel, andlight exposure time of a second pixel that differs in light exposuretime from the first pixel in such a manner that at least one of lightexposure start time or light exposure end time differs according tolight emission time of predetermined light in the pixel array unit. 2.The solid-state image pickup device according to claim 1, wherein thelight exposure control unit controls the light exposure time of thefirst pixel and the light exposure time of the second pixel in such amanner that the light exposure start time differs according to the lightemission time of the predetermined light.
 3. The solid-state imagepickup device according to claim 1, wherein the light exposure controlunit controls the light exposure time of the first pixel and the lightexposure time of the second pixel in such a manner that the lightexposure start time differs by changing an inclusion ratio of the lightemission time of the predetermined light.
 4. The solid-state imagepickup device according to claim 1, wherein the pixel includes a PD. 5.The solid-state image pickup device according to claim 4, wherein thelight exposure control unit controls the light exposure start time byreset operation, and controls the light exposure end time by chargetransfer.
 6. The solid-state image pickup device according to claim 1,wherein the pixel includes an organic or inorganic photoelectricconversion film.
 7. The solid-state image pickup device according toclaim 6, wherein the light exposure control unit controls the lightexposure start time by reset operation, and controls the light exposureend time by an upper electrode of the photoelectric conversion film. 8.The solid-state image pickup device according to claim 6, wherein thelight exposure control unit controls the light exposure start time of atleast one of the first pixel or the second pixel by an upper electrodeof the photoelectric conversion film, and controls the light exposureend time by the upper electrode of the photoelectric conversion film. 9.The solid-state image pickup device according to claim 1, wherein thelight exposure control unit controls the light exposure time of thefirst pixel and the light exposure time of the second pixel in such amanner that at least one of the light exposure start time or the lightexposure end time differs according to light emission times of aplurality of the predetermined lights.
 10. The solid-state image pickupdevice according to claim 1, wherein the light exposure control unitcontrols the light exposure time of the first pixel and the lightexposure time of the second pixel in such a manner that at least one ofthe light exposure start time or the light exposure end time differs bychanging inclusion ratios of the light emission times of a plurality ofthe predetermined lights respectively.
 11. The solid-state image pickupdevice according to claim 1, further comprising a computation unit thatsubjects images from the first pixel and the second pixel to mosaicprocessing, and performs computation processing on a pixel basis. 12.The solid-state image pickup device according to claim 1, wherein thelight exposure control unit controls the light exposure time of thefirst pixel, the light exposure time of the second pixel, and the lightexposure time of a third pixel that differs in light exposure time fromthe first pixel and the second pixel in such a manner that at least oneof the light exposure start time or the light exposure end time differsaccording to light emission times of a plurality of the predeterminedlights.
 13. The solid-state image pickup device according to claim 12,wherein the light exposure control unit controls the light exposure timeof the first pixel, the light exposure time of the second pixel, and thelight exposure time of the third pixel in such a manner that at leastone of the light exposure start time or the light exposure end timediffers by changing inclusion ratios of the light emission times of theplurality of predetermined lights respectively.
 14. The image pickupdevice according to claim 12, further comprising a computation unit thatsubjects images from the first pixel, the second pixel, and the thirdpixel to mosaic processing, and performs computation processing on apixel basis.
 15. The solid-state image pickup device according to claim1, wherein the pixel array unit includes a pixel having a conversionefficiency adjustable function.
 16. An image pickup method comprisingthe step of controlling, by a solid-state image pickup device, lightexposure time of a first pixel, and light exposure time of a secondpixel that differs in light exposure time from the first pixel in such amanner that at least one of light exposure start time or light exposureend time differs according to light emission time of predetermined lightin a pixel array unit on which pixels are two-dimensionally arranged.17. An electronic apparatus comprising: a light-emitting unit that emitslight; and a solid-state image pickup device, the solid-state imagepickup device including: a pixel array unit on which pixels aretwo-dimensionally arranged; and a light exposure control unit thatcontrols light exposure time of a first pixel, and light exposure timeof a second pixel that differs in light exposure time from the firstpixel in such a manner that at least one of light exposure start time orlight exposure end time differs according to light emission time oflight emitted by the light-emitting unit in the pixel array unit.